If we compare the absolute deviation across method pairs and whether or not SAI holds for a model, the following plot shows that for models for which SAI does not hold (i.e., the pair clustering model) the absolute deviation is on average larger. The red points is the mean in each plot. Consequently, all following analyses are based on the data excluding the non SAI (i.e., pair clustering) model.
| model2 | dataset | parameter | cond_x | cond_y | abs_dev | se_c | rhos_max | x | y | prop_ns_noasy | prop_ns_trait | prop_ns_nopb | prop_ns_nonpb | prop_ns | z_abs_dev |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| hb | Bayen2006Exp1.csv:hb | hb:c | NP-BAYES | CP-MLE | 0.189 | 0.021 | 0.034 | 0.200 | 0.011 | 0.038 | 0.000 | 0.038 | 0.000 | 0.019 | 8.915 |
| pm | Smith & Hunt 2013.csv:pm | pm:M | NP-BAYES | CP-MLE | 0.159 | 0.019 | 0.081 | 0.831 | 0.991 | 0.000 | 0.000 | 0.200 | 0.160 | 0.090 | 8.501 |
| rm | Data from Filevich Horn Kuehn 2017_within2.csv:rm | rm:r | NP-BAYES | CP-MLE | 0.091 | 0.013 | 0.025 | 0.390 | 0.300 | 0.182 | 0.101 | 0.141 | 0.000 | 0.106 | 7.065 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-BAYES | CP-MLE | 0.053 | 0.009 | 0.047 | 0.070 | 0.018 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 5.906 |
| quad | CalanchiniEtAl2014_PI_ableMCMC:quad | quad:D1 | PP-B | CP-MLE | 0.035 | 0.007 | 0.091 | 0.887 | 0.852 | 0.230 | 0.060 | 0.190 | 0.025 | 0.126 | 5.028 |
| rm | Data from Filevich Horn Kuehn 2017_within2.csv:rm | rm:r | PP-LT-C | CP-MLE | 0.106 | 0.021 | 0.025 | 0.406 | 0.300 | 0.182 | 0.101 | 0.141 | 0.000 | 0.106 | 4.994 |
| pm | Smithetal2014_Exp2.csv:pm | pm:C2 | NP-BAYES | CP-MLE | 0.059 | 0.013 | 0.071 | 0.837 | 0.895 | 0.000 | 0.000 | 0.067 | 0.022 | 0.022 | 4.400 |
| rm | Pohl et al 2017.csv:rm | rm:r | NP-BAYES | CP-MLE | 0.039 | 0.010 | 0.007 | 0.789 | 0.828 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 3.815 |
| rm | Pohl et al 2017.csv:rm | rm:r | PP-B | CP-MLE | 0.054 | 0.016 | 0.007 | 0.882 | 0.828 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 3.455 |
| rm | Pohl et al 2017.csv:rm | rm:r | PP-LT-NC | CP-MLE | 0.053 | 0.016 | 0.007 | 0.881 | 0.828 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 3.398 |
| rm | Data from Filevich Horn Kuehn 2017_within2.csv:rm | rm:r | NP-MLE | CP-MLE | 0.078 | 0.024 | 0.025 | 0.377 | 0.300 | 0.182 | 0.101 | 0.141 | 0.000 | 0.106 | 3.177 |
| rm | Data from Filevich Horn Kuehn 2017_within2.csv:rm | rm:r | NP-PB | CP-MLE | 0.078 | 0.024 | 0.025 | 0.377 | 0.300 | 0.182 | 0.101 | 0.141 | 0.000 | 0.106 | 3.177 |
| rm | Data from Filevich Horn Kuehn 2017_within2.csv:rm | rm:r | NP-NPB | CP-MLE | 0.078 | 0.024 | 0.025 | 0.377 | 0.300 | 0.182 | 0.101 | 0.141 | 0.000 | 0.106 | 3.177 |
| hb | Bayen2006Exp1.csv:hb | hb:c | PP-LT-C | CP-MLE | 0.017 | 0.005 | 0.034 | 0.028 | 0.011 | 0.038 | 0.000 | 0.038 | 0.000 | 0.019 | 3.095 |
| rm | HilbigPohl09exp3.csv:rm | rm:r | NP-BAYES | CP-MLE | 0.044 | 0.015 | 0.086 | 0.716 | 0.760 | 0.048 | 0.016 | 0.032 | 0.000 | 0.024 | 2.956 |
| rm | Horn Ruggeri Pachur 2016 .csv:rm | rm:r | NP-BAYES | CP-MLE | 0.066 | 0.024 | 0.086 | 0.327 | 0.262 | 0.167 | 0.056 | 0.111 | 0.000 | 0.083 | 2.754 |
| quad | CalanchiniEtAl2014_PI_ableMCMC:quad | quad:D1 | NP-MLE | CP-MLE | 0.020 | 0.007 | 0.091 | 0.872 | 0.852 | 0.230 | 0.060 | 0.190 | 0.025 | 0.126 | 2.754 |
| quad | CalanchiniEtAl2014_PI_ableMCMC:quad | quad:D1 | NP-PB | CP-MLE | 0.020 | 0.007 | 0.091 | 0.872 | 0.852 | 0.230 | 0.060 | 0.190 | 0.025 | 0.126 | 2.731 |
| quad | CalanchiniEtAl2014_PI_ableMCMC:quad | quad:D1 | NP-NPB | CP-MLE | 0.020 | 0.007 | 0.091 | 0.872 | 0.852 | 0.230 | 0.060 | 0.190 | 0.025 | 0.126 | 2.731 |
| quad | JinEtAl2016s2:quad | quad:ACbb1 | NP-BAYES | CP-MLE | 0.046 | 0.017 | 0.095 | 0.133 | 0.086 | 0.130 | 0.000 | 0.000 | 0.000 | 0.033 | 2.697 |
| rm | Michalkiewicz Erdfelder 2016_Exp.2_week group_test1.csv:rm | rm:r | NP-BAYES | CP-MLE | 0.030 | 0.011 | 0.035 | 0.574 | 0.604 | 0.071 | 0.048 | 0.071 | 0.000 | 0.048 | 2.636 |
| quad | GonsalkoraleEtAl2011:quad | quad:ACbb1 | NP-BAYES | CP-MLE | 0.046 | 0.019 | 0.095 | 0.136 | 0.089 | 0.174 | 0.043 | 0.000 | 0.000 | 0.054 | 2.467 |
| pm | Smith & Hunt 2013.csv:pm | pm:M | PP-LT-C | CP-MLE | 0.030 | 0.012 | 0.081 | 0.961 | 0.991 | 0.000 | 0.000 | 0.200 | 0.160 | 0.090 | 2.416 |
| pm | Smithetal2014_Exp2.csv:pm | pm:C1 | NP-BAYES | CP-MLE | 0.045 | 0.019 | 0.095 | 0.664 | 0.710 | 0.000 | 0.000 | 0.067 | 0.044 | 0.028 | 2.342 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | PP-LT-C | CP-MLE | 0.024 | 0.011 | 0.047 | 0.042 | 0.018 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 2.300 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-MLE | CP-MLE | 0.025 | 0.011 | 0.047 | 0.042 | 0.018 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 2.160 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-PB | CP-MLE | 0.025 | 0.011 | 0.047 | 0.042 | 0.018 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 2.160 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-NPB | CP-MLE | 0.025 | 0.011 | 0.047 | 0.042 | 0.018 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 2.160 |
| rm | Pohl, Michalkiewicz, Erdfelder, Hilbig 2017_Exp1.csv:rm | rm:r | NP-BAYES | CP-MLE | 0.016 | 0.009 | 0.019 | 0.606 | 0.622 | 0.172 | 0.086 | 0.155 | 0.000 | 0.103 | 1.894 |
| rm | HilbigPohl09exp2.csv:rm | rm:a | NP-BAYES | CP-MLE | 0.011 | 0.006 | 0.066 | 0.785 | 0.796 | 0.097 | 0.042 | 0.097 | 0.014 | 0.062 | 1.885 |
| model2 | dataset | parameter | cond_x | cond_y | abs_dev | se_c | rhos_max | x | y | prop_ns_noasy | prop_ns_trait | prop_ns_nopb | prop_ns_nonpb | prop_ns | z_abs_dev |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| rm | Data from Filevich Horn Kuehn 2017_within2.csv:rm | rm:r | NP-BAYES | CP-MLE | 0.091 | 0.013 | 0.025 | 0.390 | 0.300 | 0.182 | 0.101 | 0.141 | 0.000 | 0.106 | 7.065 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-BAYES | CP-MLE | 0.053 | 0.009 | 0.047 | 0.070 | 0.018 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 5.906 |
| rm | Pohl et al 2017.csv:rm | rm:r | NP-BAYES | CP-MLE | 0.039 | 0.010 | 0.007 | 0.789 | 0.828 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 3.815 |
| rm | Pohl et al 2017.csv:rm | rm:r | PP-B | CP-MLE | 0.054 | 0.016 | 0.007 | 0.882 | 0.828 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 3.455 |
| rm | Pohl et al 2017.csv:rm | rm:r | PP-LT-NC | CP-MLE | 0.053 | 0.016 | 0.007 | 0.881 | 0.828 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 3.398 |
| hb | Bayen2006Exp1.csv:hb | hb:c | PP-LT-C | CP-MLE | 0.017 | 0.005 | 0.034 | 0.028 | 0.011 | 0.038 | 0.000 | 0.038 | 0.000 | 0.019 | 3.095 |
| rm | Michalkiewicz Erdfelder 2016_Exp.2_week group_test1.csv:rm | rm:r | NP-BAYES | CP-MLE | 0.030 | 0.011 | 0.035 | 0.574 | 0.604 | 0.071 | 0.048 | 0.071 | 0.000 | 0.048 | 2.636 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | PP-LT-C | CP-MLE | 0.024 | 0.011 | 0.047 | 0.042 | 0.018 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 2.300 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-MLE | CP-MLE | 0.025 | 0.011 | 0.047 | 0.042 | 0.018 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 2.160 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-PB | CP-MLE | 0.025 | 0.011 | 0.047 | 0.042 | 0.018 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 2.160 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-NPB | CP-MLE | 0.025 | 0.011 | 0.047 | 0.042 | 0.018 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 2.160 |
| rm | Pohl, Michalkiewicz, Erdfelder, Hilbig 2017_Exp1.csv:rm | rm:r | NP-BAYES | CP-MLE | 0.016 | 0.009 | 0.019 | 0.606 | 0.622 | 0.172 | 0.086 | 0.155 | 0.000 | 0.103 | 1.894 |
| quad | JinEtAl2016s2:quad | quad:ACbb1 | NP-BAYES | CP-MLE | 0.031 | 0.017 | 0.044 | 0.156 | 0.126 | 0.034 | 0.000 | 0.000 | 0.000 | 0.009 | 1.809 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:A | NP-BAYES | CP-MLE | 0.010 | 0.006 | 0.047 | 0.244 | 0.254 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 1.695 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | NP-BAYES | CP-MLE | 0.031 | 0.019 | 0.035 | 0.031 | 0.000 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 1.668 |
| rm | HilbigPohl09exp2.csv:rm | rm:b | NP-BAYES | CP-MLE | 0.012 | 0.007 | 0.029 | 0.671 | 0.684 | 0.097 | 0.042 | 0.097 | 0.014 | 0.062 | 1.662 |
| rm | Michalkiewicz Erdfelder 2016_Exp.2_week group_test2.csv:rm | rm:r | NP-BAYES | CP-MLE | 0.015 | 0.009 | 0.028 | 0.713 | 0.728 | 0.190 | 0.071 | 0.190 | 0.000 | 0.113 | 1.634 |
| pm | SmithBayen2005_Experiment2.csv:pm | pm:C1 | NP-BAYES | CP-MLE | 0.025 | 0.020 | 0.030 | 0.834 | 0.860 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 1.302 |
| 2htsm_4 | S2002_E2:2htsm_4 | 2htsm_4:D | NP-BAYES | CP-MLE | 0.015 | 0.013 | 0.042 | 0.904 | 0.919 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 1.137 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | PP-B | CP-MLE | 0.010 | 0.009 | 0.047 | 0.008 | 0.018 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 1.111 |
| rm | Pohl, Michalkiewicz, Erdfelder, Hilbig 2017_Exp1.csv:rm | rm:a | NP-BAYES | CP-MLE | 0.006 | 0.006 | 0.043 | 0.528 | 0.522 | 0.172 | 0.086 | 0.155 | 0.000 | 0.103 | 1.010 |
| hb | Bayen2006Exp1.csv:hb | hb:c | NP-MLE | CP-MLE | 0.004 | 0.004 | 0.034 | 0.008 | 0.011 | 0.038 | 0.000 | 0.038 | 0.000 | 0.019 | 1.003 |
| hb | Bayen2006Exp1.csv:hb | hb:c | NP-PB | CP-MLE | 0.004 | 0.004 | 0.034 | 0.008 | 0.011 | 0.038 | 0.000 | 0.038 | 0.000 | 0.019 | 1.003 |
| hb | Bayen2006Exp1.csv:hb | hb:c | NP-NPB | CP-MLE | 0.004 | 0.004 | 0.034 | 0.008 | 0.011 | 0.038 | 0.000 | 0.038 | 0.000 | 0.019 | 1.003 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | PP-LT-C | CP-MLE | 0.016 | 0.017 | 0.035 | 0.016 | 0.000 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 0.933 |
| rm | Hilbig et al 2015_exp3.csv:rm | rm:a | NP-BAYES | CP-MLE | 0.013 | 0.014 | 0.017 | 0.690 | 0.703 | 0.062 | 0.000 | 0.062 | 0.000 | 0.031 | 0.932 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | PP-LT-NC | CP-MLE | 0.008 | 0.009 | 0.047 | 0.009 | 0.018 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 0.928 |
| rm | HilbigPohl09exp2.csv:rm | rm:b | NP-MLE | CP-MLE | 0.008 | 0.009 | 0.029 | 0.675 | 0.684 | 0.097 | 0.042 | 0.097 | 0.014 | 0.062 | 0.905 |
| rm | HilbigPohl09exp2.csv:rm | rm:b | NP-PB | CP-MLE | 0.008 | 0.009 | 0.029 | 0.675 | 0.684 | 0.097 | 0.042 | 0.097 | 0.014 | 0.062 | 0.905 |
| rm | HilbigPohl09exp2.csv:rm | rm:b | NP-NPB | CP-MLE | 0.008 | 0.009 | 0.029 | 0.675 | 0.684 | 0.097 | 0.042 | 0.097 | 0.014 | 0.062 | 0.905 |
| model2 | dataset | parameter | cond_x | cond_y | abs_dev | se_c | rhos_max | x | y | prop_ns_noasy | prop_ns_trait | prop_ns_nopb | prop_ns_nonpb | prop_ns | z_abs_dev |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| hb | Coolin2015.csv:hb | hb:c | NP-BAYES | PP-LT-C | 0.230 | 0.017 | 0.061 | 0.248 | 0.018 | 0.047 | 0.031 | 0.094 | 0.000 | 0.043 | 13.263 |
| hb | Coolin2015.csv:hb | hb:c | NP-MLE | PP-LT-C | 0.018 | 0.003 | 0.061 | 0.000 | 0.018 | 0.047 | 0.031 | 0.094 | 0.000 | 0.043 | 6.593 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | NP-MLE | PP-LT-C | 0.016 | 0.002 | 0.035 | 0.000 | 0.016 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 6.504 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | NP-PB | PP-LT-C | 0.016 | 0.002 | 0.035 | 0.000 | 0.016 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 6.504 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | NP-NPB | PP-LT-C | 0.016 | 0.002 | 0.035 | 0.000 | 0.016 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 6.504 |
| pm | Smith & Hunt 2013.csv:pm | pm:M | NP-BAYES | PP-LT-C | 0.129 | 0.022 | 0.081 | 0.831 | 0.961 | 0.000 | 0.000 | 0.200 | 0.160 | 0.090 | 5.954 |
| rm | Data from Filevich Horn Kuehn 2017_within2.csv:rm | rm:r | CP-Bayes | PP-LT-C | 0.106 | 0.021 | 0.025 | 0.300 | 0.406 | 0.182 | 0.101 | 0.141 | 0.000 | 0.106 | 5.026 |
| rm | Data from Filevich Horn Kuehn 2017_within2.csv:rm | rm:r | CP-MLE | PP-LT-C | 0.106 | 0.021 | 0.025 | 0.300 | 0.406 | 0.182 | 0.101 | 0.141 | 0.000 | 0.106 | 4.994 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | PP-LT-NC | PP-LT-C | 0.015 | 0.003 | 0.035 | 0.001 | 0.016 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 4.708 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | CP-Bayes | PP-LT-C | 0.015 | 0.003 | 0.035 | 0.001 | 0.016 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 4.565 |
| hb | Coolin2015.csv:hb | hb:c | PP-LT-NC | PP-LT-C | 0.016 | 0.004 | 0.061 | 0.002 | 0.018 | 0.047 | 0.031 | 0.094 | 0.000 | 0.043 | 4.097 |
| hb | Coolin2015.csv:hb | hb:c | PP-B | PP-LT-C | 0.016 | 0.004 | 0.061 | 0.002 | 0.018 | 0.047 | 0.031 | 0.094 | 0.000 | 0.043 | 3.867 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | PP-B | PP-LT-C | 0.034 | 0.009 | 0.047 | 0.008 | 0.042 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 3.857 |
| hb | Coolin2015.csv:hb | hb:c | CP-Bayes | PP-LT-C | 0.015 | 0.004 | 0.061 | 0.003 | 0.018 | 0.047 | 0.031 | 0.094 | 0.000 | 0.043 | 3.773 |
| pm | Smithetal2014_Exp2.csv:pm | pm:C2 | NP-BAYES | PP-LT-C | 0.055 | 0.015 | 0.071 | 0.837 | 0.891 | 0.000 | 0.000 | 0.067 | 0.022 | 0.022 | 3.643 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | PP-LT-NC | PP-LT-C | 0.033 | 0.009 | 0.047 | 0.009 | 0.042 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 3.624 |
| quad | CalanchiniEtAl2014_PI_ableMCMC:quad | quad:D1 | PP-B | PP-LT-C | 0.033 | 0.009 | 0.091 | 0.887 | 0.854 | 0.230 | 0.060 | 0.190 | 0.025 | 0.126 | 3.522 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-BAYES | PP-LT-C | 0.028 | 0.009 | 0.047 | 0.070 | 0.042 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 3.188 |
| hb | Bayen2006Exp1.csv:hb | hb:c | CP-MLE | PP-LT-C | 0.017 | 0.005 | 0.034 | 0.011 | 0.028 | 0.038 | 0.000 | 0.038 | 0.000 | 0.019 | 3.095 |
| rm | Pohl et al 2017.csv:rm | rm:r | PP-B | PP-LT-C | 0.057 | 0.020 | 0.007 | 0.882 | 0.825 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 2.881 |
| rm | Pohl et al 2017.csv:rm | rm:r | PP-LT-NC | PP-LT-C | 0.056 | 0.020 | 0.007 | 0.881 | 0.825 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 2.836 |
| hb | Bayen2006Exp1.csv:hb | hb:c | NP-MLE | PP-LT-C | 0.020 | 0.008 | 0.034 | 0.008 | 0.028 | 0.038 | 0.000 | 0.038 | 0.000 | 0.019 | 2.670 |
| hb | Bayen2006Exp1.csv:hb | hb:c | NP-PB | PP-LT-C | 0.020 | 0.008 | 0.034 | 0.008 | 0.028 | 0.038 | 0.000 | 0.038 | 0.000 | 0.019 | 2.670 |
| hb | Bayen2006Exp1.csv:hb | hb:c | NP-NPB | PP-LT-C | 0.020 | 0.008 | 0.034 | 0.008 | 0.028 | 0.038 | 0.000 | 0.038 | 0.000 | 0.019 | 2.670 |
| pm | Smith & Hunt 2013.csv:pm | pm:M | NP-MLE | PP-LT-C | 0.031 | 0.012 | 0.081 | 0.991 | 0.961 | 0.000 | 0.000 | 0.200 | 0.160 | 0.090 | 2.540 |
| rm | Pohl et al 2017.csv:rm | rm:r | NP-BAYES | PP-LT-C | 0.036 | 0.014 | 0.007 | 0.789 | 0.825 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 2.521 |
| pm | Smith & Hunt 2013.csv:pm | pm:M | CP-MLE | PP-LT-C | 0.030 | 0.012 | 0.081 | 0.991 | 0.961 | 0.000 | 0.000 | 0.200 | 0.160 | 0.090 | 2.416 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | NP-BAYES | PP-LT-C | 0.015 | 0.007 | 0.035 | 0.031 | 0.016 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 2.327 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | CP-MLE | PP-LT-C | 0.024 | 0.011 | 0.047 | 0.018 | 0.042 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 2.300 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | CP-Bayes | PP-LT-C | 0.023 | 0.010 | 0.047 | 0.019 | 0.042 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 2.300 |
| model2 | dataset | parameter | cond_x | cond_y | abs_dev | se_c | rhos_max | x | y | prop_ns_noasy | prop_ns_trait | prop_ns_nopb | prop_ns_nonpb | prop_ns | z_abs_dev |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | NP-MLE | PP-LT-C | 0.016 | 0.002 | 0.035 | 0.000 | 0.016 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 6.504 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | NP-PB | PP-LT-C | 0.016 | 0.002 | 0.035 | 0.000 | 0.016 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 6.504 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | NP-NPB | PP-LT-C | 0.016 | 0.002 | 0.035 | 0.000 | 0.016 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 6.504 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | PP-LT-NC | PP-LT-C | 0.015 | 0.003 | 0.035 | 0.001 | 0.016 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 4.708 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | CP-Bayes | PP-LT-C | 0.015 | 0.003 | 0.035 | 0.001 | 0.016 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 4.565 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | PP-B | PP-LT-C | 0.034 | 0.009 | 0.047 | 0.008 | 0.042 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 3.857 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | PP-LT-NC | PP-LT-C | 0.033 | 0.009 | 0.047 | 0.009 | 0.042 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 3.624 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-BAYES | PP-LT-C | 0.028 | 0.009 | 0.047 | 0.070 | 0.042 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 3.188 |
| hb | Bayen2006Exp1.csv:hb | hb:c | CP-MLE | PP-LT-C | 0.017 | 0.005 | 0.034 | 0.011 | 0.028 | 0.038 | 0.000 | 0.038 | 0.000 | 0.019 | 3.095 |
| rm | Pohl et al 2017.csv:rm | rm:r | PP-B | PP-LT-C | 0.057 | 0.020 | 0.007 | 0.882 | 0.825 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 2.881 |
| rm | Pohl et al 2017.csv:rm | rm:r | PP-LT-NC | PP-LT-C | 0.056 | 0.020 | 0.007 | 0.881 | 0.825 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 2.836 |
| hb | Bayen2006Exp1.csv:hb | hb:c | NP-MLE | PP-LT-C | 0.020 | 0.008 | 0.034 | 0.008 | 0.028 | 0.038 | 0.000 | 0.038 | 0.000 | 0.019 | 2.670 |
| hb | Bayen2006Exp1.csv:hb | hb:c | NP-PB | PP-LT-C | 0.020 | 0.008 | 0.034 | 0.008 | 0.028 | 0.038 | 0.000 | 0.038 | 0.000 | 0.019 | 2.670 |
| hb | Bayen2006Exp1.csv:hb | hb:c | NP-NPB | PP-LT-C | 0.020 | 0.008 | 0.034 | 0.008 | 0.028 | 0.038 | 0.000 | 0.038 | 0.000 | 0.019 | 2.670 |
| rm | Pohl et al 2017.csv:rm | rm:r | NP-BAYES | PP-LT-C | 0.036 | 0.014 | 0.007 | 0.789 | 0.825 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 2.521 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | NP-BAYES | PP-LT-C | 0.015 | 0.007 | 0.035 | 0.031 | 0.016 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 2.327 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | CP-MLE | PP-LT-C | 0.024 | 0.011 | 0.047 | 0.018 | 0.042 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 2.300 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | CP-Bayes | PP-LT-C | 0.023 | 0.010 | 0.047 | 0.019 | 0.042 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 2.300 |
| hb | Bayen2006Exp1.csv:hb | hb:c | PP-B | PP-LT-C | 0.019 | 0.009 | 0.034 | 0.009 | 0.028 | 0.038 | 0.000 | 0.038 | 0.000 | 0.019 | 2.152 |
| hb | Bayen2006Exp1.csv:hb | hb:c | CP-Bayes | PP-LT-C | 0.014 | 0.009 | 0.034 | 0.014 | 0.028 | 0.038 | 0.000 | 0.038 | 0.000 | 0.019 | 1.655 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | PP-B | PP-LT-C | 0.014 | 0.009 | 0.035 | 0.002 | 0.016 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 1.593 |
| quad | JinEtAl2016s2:quad | quad:ACbb1 | NP-BAYES | PP-LT-C | 0.028 | 0.019 | 0.044 | 0.156 | 0.128 | 0.034 | 0.000 | 0.000 | 0.000 | 0.009 | 1.512 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | CP-MLE | PP-LT-C | 0.016 | 0.017 | 0.035 | 0.000 | 0.016 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 0.933 |
| 2htsm_4 | S2002_E2:2htsm_4 | 2htsm_4:D | CP-MLE | PP-LT-C | 0.015 | 0.018 | 0.042 | 0.919 | 0.904 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.811 |
| rm | Data from Filevich Horn Kuehn 2017_within2.csv:rm | rm:r | NP-BAYES | PP-LT-C | 0.015 | 0.019 | 0.025 | 0.390 | 0.406 | 0.182 | 0.101 | 0.141 | 0.000 | 0.106 | 0.808 |
| 2htsm_4 | S2002_E2:2htsm_4 | 2htsm_4:D | NP-MLE | PP-LT-C | 0.015 | 0.020 | 0.042 | 0.919 | 0.904 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.750 |
| 2htsm_4 | S2002_E2:2htsm_4 | 2htsm_4:D | NP-PB | PP-LT-C | 0.015 | 0.020 | 0.042 | 0.919 | 0.904 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.750 |
| 2htsm_4 | S2002_E2:2htsm_4 | 2htsm_4:D | NP-NPB | PP-LT-C | 0.015 | 0.020 | 0.042 | 0.919 | 0.904 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.750 |
| 2htsm_4 | S2002_E2:2htsm_4 | 2htsm_4:D | CP-Bayes | PP-LT-C | 0.013 | 0.018 | 0.042 | 0.917 | 0.904 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.711 |
| pm | SmithBayen2005_Experiment2.csv:pm | pm:C1 | NP-MLE | PP-LT-C | 0.013 | 0.019 | 0.030 | 0.860 | 0.846 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.685 |
| model2 | dataset | parameter | cond_x | cond_y | abs_dev | se_c | rhos_max | x | y | prop_ns_noasy | prop_ns_trait | prop_ns_nopb | prop_ns_nonpb | prop_ns | z_abs_dev |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| hb | Coolin2015.csv:hb | hb:c | NP-BAYES | PP-LT-C | 0.230 | 0.017 | 0.061 | 0.248 | 0.018 | 0.047 | 0.031 | 0.094 | 0.000 | 0.043 | 13.263 |
| 2htsm_5d | Mieth et al. (2016) Ex2:2htsm_5d | 2htsm_5d:d_2 | NP-BAYES | PP-LT-C | 0.311 | 0.043 | 0.058 | 0.437 | 0.126 | 0.281 | 0.010 | 0.042 | 0.000 | 0.083 | 7.161 |
| 2htsm_6e | Bell et al. (2015) Ex2:2htsm_6e | 2htsm_6e:d_1 | NP-BAYES | PP-LT-C | 0.305 | 0.044 | 0.084 | 0.448 | 0.143 | 0.982 | 0.009 | 0.026 | 0.000 | 0.254 | 6.861 |
| hb | Bayen2006Exp1.csv:hb | hb:c | NP-BAYES | PP-LT-C | 0.172 | 0.025 | 0.034 | 0.200 | 0.028 | 0.038 | 0.000 | 0.038 | 0.000 | 0.019 | 6.795 |
| hb | Coolin2015.csv:hb | hb:c | NP-MLE | PP-LT-C | 0.018 | 0.003 | 0.061 | 0.000 | 0.018 | 0.047 | 0.031 | 0.094 | 0.000 | 0.043 | 6.593 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | NP-MLE | PP-LT-C | 0.016 | 0.002 | 0.035 | 0.000 | 0.016 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 6.504 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | NP-PB | PP-LT-C | 0.016 | 0.002 | 0.035 | 0.000 | 0.016 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 6.504 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | NP-NPB | PP-LT-C | 0.016 | 0.002 | 0.035 | 0.000 | 0.016 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 6.504 |
| 2htsm_4 | M2003:2htsm_4 | 2htsm_4:d | NP-BAYES | PP-LT-C | 0.229 | 0.038 | 0.069 | 0.321 | 0.092 | 0.061 | 0.000 | 0.030 | 0.000 | 0.023 | 6.055 |
| pm | Smith & Hunt 2013.csv:pm | pm:M | NP-BAYES | PP-LT-C | 0.129 | 0.022 | 0.081 | 0.831 | 0.961 | 0.000 | 0.000 | 0.200 | 0.160 | 0.090 | 5.954 |
| 2htsm_4 | M2003:2htsm_4 | 2htsm_4:d | NP-BAYES | PP-LT-C | 0.224 | 0.039 | 0.070 | 0.327 | 0.103 | 0.061 | 0.000 | 0.030 | 0.000 | 0.023 | 5.783 |
| rm | Data from Filevich Horn Kuehn 2017_within2.csv:rm | rm:r | CP-Bayes | PP-LT-C | 0.106 | 0.021 | 0.025 | 0.300 | 0.406 | 0.182 | 0.101 | 0.141 | 0.000 | 0.106 | 5.026 |
| rm | Data from Filevich Horn Kuehn 2017_within2.csv:rm | rm:r | CP-MLE | PP-LT-C | 0.106 | 0.021 | 0.025 | 0.300 | 0.406 | 0.182 | 0.101 | 0.141 | 0.000 | 0.106 | 4.994 |
| 2htsm_4 | MH2001:2htsm_4 | 2htsm_4:d | NP-BAYES | PP-LT-C | 0.182 | 0.037 | 0.079 | 0.253 | 0.071 | 0.100 | 0.000 | 0.000 | 0.000 | 0.025 | 4.860 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | PP-LT-NC | PP-LT-C | 0.015 | 0.003 | 0.035 | 0.001 | 0.016 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 4.708 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | CP-Bayes | PP-LT-C | 0.015 | 0.003 | 0.035 | 0.001 | 0.016 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 4.565 |
| 2htsm_4 | MH2001:2htsm_4 | 2htsm_4:d | NP-BAYES | PP-LT-C | 0.191 | 0.043 | 0.077 | 0.285 | 0.093 | 0.100 | 0.000 | 0.000 | 0.000 | 0.025 | 4.443 |
| 2htsm_4 | BK2011_E1:2htsm_4 | 2htsm_4:b | NP-BAYES | PP-LT-C | 0.200 | 0.046 | 0.084 | 0.330 | 0.130 | 0.042 | 0.000 | 0.083 | 0.000 | 0.031 | 4.327 |
| rm | Data from Filevich Horn Kuehn 2017_within2.csv:rm | rm:r | PP-LT-NC | PP-LT-C | 0.176 | 0.041 | 0.025 | 0.229 | 0.406 | 0.182 | 0.101 | 0.141 | 0.000 | 0.106 | 4.260 |
| rm | Data from Filevich Horn Kuehn 2017_within2.csv:rm | rm:r | PP-B | PP-LT-C | 0.179 | 0.042 | 0.025 | 0.227 | 0.406 | 0.182 | 0.101 | 0.141 | 0.000 | 0.106 | 4.254 |
| hb | Coolin2015.csv:hb | hb:c | PP-LT-NC | PP-LT-C | 0.016 | 0.004 | 0.061 | 0.002 | 0.018 | 0.047 | 0.031 | 0.094 | 0.000 | 0.043 | 4.097 |
| hb | Coolin2015.csv:hb | hb:c | PP-B | PP-LT-C | 0.016 | 0.004 | 0.061 | 0.002 | 0.018 | 0.047 | 0.031 | 0.094 | 0.000 | 0.043 | 3.867 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | PP-B | PP-LT-C | 0.034 | 0.009 | 0.047 | 0.008 | 0.042 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 3.857 |
| hb | Coolin2015.csv:hb | hb:c | CP-Bayes | PP-LT-C | 0.015 | 0.004 | 0.061 | 0.003 | 0.018 | 0.047 | 0.031 | 0.094 | 0.000 | 0.043 | 3.773 |
| hb | GroßBayen2015.csv:hb | hb:b | NP-BAYES | PP-LT-C | 0.112 | 0.030 | 0.048 | 0.299 | 0.411 | 0.022 | 0.022 | 0.022 | 0.000 | 0.016 | 3.717 |
| pm | Smithetal2014_Exp2.csv:pm | pm:C2 | NP-BAYES | PP-LT-C | 0.055 | 0.015 | 0.071 | 0.837 | 0.891 | 0.000 | 0.000 | 0.067 | 0.022 | 0.022 | 3.643 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | PP-LT-NC | PP-LT-C | 0.033 | 0.009 | 0.047 | 0.009 | 0.042 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 3.624 |
| quad | CalanchiniEtAl2014_PI_ableMCMC:quad | quad:D1 | PP-B | PP-LT-C | 0.033 | 0.009 | 0.091 | 0.887 | 0.854 | 0.230 | 0.060 | 0.190 | 0.025 | 0.126 | 3.522 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-BAYES | PP-LT-C | 0.028 | 0.009 | 0.047 | 0.070 | 0.042 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 3.188 |
| 2htsm_4 | BK2011_E1:2htsm_4 | 2htsm_4:b | NP-PB | PP-LT-C | 0.106 | 0.034 | 0.084 | 0.023 | 0.130 | 0.042 | 0.000 | 0.083 | 0.000 | 0.031 | 3.164 |